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This  program  of  research  and  development  is  directed  at  the  systematic 
design,  implementation,  and  evaluation  of  innovative  graphic  concepts  for 
supporting  tactical  decision  making.  Essentially,  the  idea  is  to  identify 
and  exploit  the  capability  of  computer-generated  situation  displays  in  a 
manner  that  can  best  serve  the  information  requirements  and  processing 
demands  of  tactical  users.  The  program  focuses  on  new  ways  of  dynamically 
portraying  and  coding  tactical  symbols  (e.g.,  use  of  color,  projection 
vectors,  perimeter  density,  etc.)  so  that  relevant  task-based  performance 
can  be  significantly  enhanced. x During  the  report  period,  a  preliminary 
study  was  conducted  as  described  in  detail  below. 


Preliminary  Studv 


Introduction.  Traditionally,  battlefield  situation  displays  have  consisted 
of  acetate  sheets,  annotated  with  grease  pencils,  and  overlayed  on  topo- 
graphic  maps.  Such  displays  have  typically  been  information  intensive  and  (  \\  \ 
full  of  clutter,  especially  in  that  certain  "givens"  like  unit  identification  . 
information  are  usually  maintained  in  constant  view.  Since  both  intuition  M 

and  empirical  data  suggest  that  the  portrayal  of  large  amounts  of  graphic 
information  in  a  tactical  display  can  retard  analytical  judgments,  the 
reduction  of  unnecessary  symbol  detail  and  display  clutter  is  likely  to  j 
enhance  information  integration.  One  way  of  achieving  such  reductions  tT 

in  tactical  displays  is  through  the  use  of  selective  (adaptive),  call 
of  only  task-relevant  core  symbols  (units)  and/or  appended  symbol 


information  (e.g.,  unit  capability,  strength,  range,  etc.).  In  thisA.  ,3f ******>"»' 
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manner,  the  dynamic  storage  and  graphic  generation  capabilities  of  an 
automated  system  can  be  exploited  to  display  only  the  Information 
necessary  for  the  accomplishment  of  a  given  task.  Based  on  the  above 
rationale,  a  preliminary  study  was  conducted  to:  (a)  examine  the  useful¬ 
ness  of  a  selective  call-up  option  for  a  variety  of  symbology  overlays  to 
be  used  in  answering  complex  tactical  questions  about  the  battlefield;  and 
(b)  provide  data  necessary  to  derive  guidelines  concerning  which  elements 
of  graphic  information  are  most  useful  In  answering  specific  tactical 
questions  that  are  likely  to  be  encountered  by  a  tactical  user. 

Two  important  prerequisites  for  this  research  were  to  decide  (1)  what 
tactical  information  is  Important  to  display  on  the  overlays,  and  (2)  how 
the  information  should  be  graphically  portrayed.  Guidance  In  making  these 
decisions  was  drawn  from  previous  work,  primarily  from  the  tactical -display 
user  community  survey  conducted  by  Landee,  Geiselman,  and  Clark  (1981). 

In  the  results  of  this  survey,  it  was  found  that  there  are  several  basic 
types  of  tactical  information  that  are  not  currently  portrayable  with 
standard  military  symbology  (FM  21-30),  namely  unit  strength  (percent  of 
full  capacity  currently  available),  unit  capability  (current  major  weapon), 
weapon  range,  and  current  direction  of  unit  movement.  Thus,  it  was  decided 
that  these  factors  should  be  included  on  the  symbology  overlays  to  be 
investigated  in  addition  to  the  units  themselves  and  their  designators. 

The  methods  of  symbology  portrayal  were  also  guided  by  the  survey  results. 

The  methods  of  personalized  graphic  portrayal  being  used  by  military  users 
in  the  field  were  found  to  be  largely  alphanumeric,  but  weapon  range 
and  direction  of  movement  arrows  were  two  notable  exceptions.  Simple 
versions  of  these  latter  methods  represent  promising  techniques  (e.g.,  Samet, 
Geiselman,  and  Landee,  1980)  and  they  were  implemented  in  the  present 
investigation.  Alphanumeric  elements  were  used  to  indicate  the  major 
weapon  capability  of  a  given  unit  and  this  information  was  placed  immediately 
below  the  unit  symbol.  Since  unit  designators  are  conventionally  portrayed 
alphanumerically,  to  either  side  of  the  unit  symbol,  alphanumerics  could  not 
be  used  to  portray  unit  strength.  Instead,  perimeter  density  coding  (number 
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of  sides  of  the  unit  symbol  blackened  In)  was  chosen,  as  other  empirical 
research  has  shown  that  perimeter  density  coding  Is  perceived  as  compatible 
with  the  concept  of  unit  strength  (Samet,  et  al.,  1980).  The  overlays  and 
their  graphic  portrayal  are  described  In  greater  detail  below. 

More  specifically,  £he  objectives  of  the  investigation  were  as  follows: 

■  ,)  (l)''  Assess  user  preferences  for  selected,  Innovative 

tactical  symbol  design  features  for  graphically 
portraying  and  communicating  different  types  of 
critical  Information  about  unit  configurations 
Including  capability,  strength,  range,  etc.  ' 

(2)  Measure  differences  in  the  speed  and  accuracy 
that  key  tactical  questions  can  be  answered  as 

a  function  of  the  display  of  a  variety  of  symbol 
overlays  which  can  be  viewed  selectively  by  the 
operator, 

(3)  Assess  user  attitudes  concerning  the  actual  and 
potential  use  and  effectiveness  of  the  automated 
selective  display  capabilities  for  serving  tactical 
mission  requirements. 

(4)  Demonstrate  a  method  for  deriving  a  set  of  initial 
procedural  guidelines  to  aid  tactical  users  in 
selecting  the  specific  displays  likely  to  be  most 
appropriate  and  useful  for  answering  particular 
tactical  questions. 

r 


3- 


Given  these  goals,  a  pilot  Investigation  was  designed  and  conducted  as 
described  In  the  subsequent  section. 

Methods 

Subjects .  The  preliminary  study  or  pilot  test  was  conducted  with  six 
Marine  Corps  personnel  serving  as  participants.  The  participants  In  the 
study  came  from  a  variety  of  background  Including  Infantry,  artillery, 
and  air  defense,  as  well  as  various  ranks  (one  Master  Sergeant,  four 
Captains,  and  one  Major).  Two  experimental  sessions  were  conducted  on 
one  day;  the  morning  session  contained  two  participants  and  the  afternoon 
had  four  participants. 

Background  Scenarios.  For  the  purposes  of  this  Investigation,  an  initial 
scenario,  and  an  updated  situation  associated  with  it  (i.e.,  a  second 
scenario),  were  prepared  with  considerable  military  realism.  The  tactical 
background  for  the  scenario  situations  are  based  on  an  Infiltration  of 
enemy  forces  Into  a  friendly  area  (in  the  vicinity  of  Camp  Pendleton); 
this  background,  and  related  situation  data,  have  been  adapted  from  Marine 
tactical  scenarios  used  In  previous  MCTSSA  experiments.  The  first  scenario 
provides  an  account  of  friendly  offensive  action  and  the  second  scenario 
or  update,  which  is  designed  to  be  given  to  the  participants  mid-way 
through  the  experimental  session,  provides  a  three-hour  extension  of  the 
tactical  situation.  This  update  describes  an  enemy  counter-attack  which 
places  the  friendly  forces  in  a  defensive  posture. 

Procedure.  Each  pilot  test  session  began  with  an  extensive  briefing  to 
participants,  which  explained  the  purpose  of  the  study  and  presented  an 
overview  of  the  experimental  procedures.  During  the  briefing,  participants 
were  familiarized  with  the  background  tactical  scenario;  then  the  overall 
test  procedure  and  the  nature  of  the  overlays  and  the  map  backgrounds  that 
would  be  used  in  the  test  were  described.  Also,  participants  were  shown 
a  sample  of  the  tactical  questions  they  would  be  asked  during  the  test  and 


received  Instructions  as  to  the  manner  In  which  the  questions  would  be 
presented  and  to  the  scope  of  answers  desired. 

Participants  were  then  shown  the  arrangement  of  their  Individual  work 
stations  consisting  of  a  keyboard,  a  monochromatic  CRT,  and  a  color  graphics 
display  monitor.  The  CRT  was  used  to  display  a  menu  of  available  overlays 
and  map  backgrounds,  by  addressing  this  menu  from  the  keyboard  the  user 
was  able  to  control  the  display  of  overlays  and  map  backgrounds  on  the 
graphics  monitor.  In  addition,  the  menu  screen  Indicated,  for  the  users 
convenience,  which  overlays  were  already  displayed  on  the  graphics  screen 
at  that  moment.  The  subjects  were  Instructed  on  how  to  use  the  menu 
features  to  call  up  and  delete  the  various  overlays  and  map  backgrounds. 

The  Instructional  procedure  Included  an  online  comprehensive  demonstration 
by  the  experimenter  of  how  each  overlay  was  to  be  selected/deleted  and 
exactly  what  the  resultant  configuration  of  overlays  would  look  like  on 
the  graphics  screen.  The  experimenter  then  went  through  a  practice 
question  by  sequentially  calling  up  and  explaining  what  he  thought  would 
be  a  set  of  appropriate  overlays  for  answering  the  question.  Following 
this  demonstration,  each  subject  was  permitted  to  experiment  by  himself 
with  the  system;  during  a  10-minute  period  he  was  encouraged  to  practice 
the  selection/deletion  of  overlays  and  map  backgrounds  and  see  how  the 
resultant  configurations  appeared  on  the  screen.  Finally,  each  subject  comp! 
ted  one  practice  problem.  Overall,  the  briefing,  demonstration,  and  practice 
sessions  lasted  a  total  of  about  90  minutes  for  each  group  of  subjects. 

After  a  short  break,  the  subjects  began  the  experimental  session.  A 
sequence  of  tactical  questions  was  prepared  in  a  different  random  order 
for  each  subject;  each  question  was  typed  out  on  a  separate  piece  of  paper 
which  also  provided  adequate  space  for  the  subject's  written  answer.  Several 
examples  of  the  tactical  questions  that  were  presented  are  given  in  Table 
1.  As  evident  from  their  content,  the  questions  represented  both  different 
echelons  of  command  and  different  tactical  functions  (e.g.,  operations 
logistics).  All  questions  were  designed  so  that  they  could  be  adequately 


TABLE  1 

SAMPLE  TACTICAL  QUESTIONS 


LABEL 


A 


B 


C 


D 


E 


F 


CONTENT 


The  Regimental  Commander  has  decided  to  commit  his  tank 
reserve.  As  a  member  of  the  operations  shop,  where  do 
you  reconnend  the  tank  reserve  be  employed?  Provide 
reasoning  for  your  answer. 

As  a  member  of  the  11th  Marines  artillery  operations  shop, 
would  you  recommend  relocating  any  of  the  artillery 
batteries?  Provide  reasoning  for  your  answer  and  also, 
the  proposed  locations  for  the  artillery  batteries  If 
you  recommend  their  relocation. 

As  a  member  of  the  operations  shop  of  2/7  (2nd  Battalion/ 
7th  Marines),  would  you  reconnend  relocating  the  command 
post?  Provide  reasoning  for  your  answer  and  also,  give 
the  proposed  location  If  you  recommend  Its  relocation. 


As  a  member  of  the  logistics  shop  of  2/7  (2nd  Battalion/ 
7th  Marines),  determine  the  priority  of  resupply  for 
2/7  units. 

As  a  member  of  the  operations  staff  of  1/7  (1st 
Battalion/7th  Marines),  would  you  recommend  committing 
the  1/7  reserves  at  this  time?  Provide  reasoning  for 
your  answer. 

As  a  member  of  the  operations  staff  of  the  7th  Marines, 
would  you  recommend  committing  the  regimental  reserves  at 
this  time?  Provide  reasoning  for  your  answer. 
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answered  on  the  basis  of  the  tactical  Information  graphically  displayed  on 
the  various  overlays.  To  pinpoint  a  geographical  location,  subjects  were 
able  to  position  a  cursor  with  a  joystick  and  then  query  the  system  to 
automatically  obtain  a  readout  of  eight  grid  coordinates. 

Each  subject  worked  at  his  own  pace  and  received  each  new  question  from  the 
experimenter  after  completing  and  submitting  the  previous  one.  Prior  to 
starting  the  subject  on  a  new  question,  the  experimenter  input  into  the 
computer  (via  the  keyboard  at  the  subject's  station)  the  Identification 
code  for  the  problem  to  be  worked  on,  and  he  also  initialized  the  journal 
record  and  timing  parameters  (l.e.,  data  collection  programs)  for  that 
subject.  In  answering  each  question,  the  subject  was  free  to  request  any 
sequence  and  configuration  of  overlays  and  map  backgrounds  and  to  maintain 
these  in  view  for  as  long  as  he  desired.  However,  across  subjects,  the 
median  completion  time  per  question  (based  on  a  total  of  27  questions)  was 
5.26  minutes  with  a  range  from  1.36  to  13.51  minutes.  Nevertheless,  since 
each  subject  worked  at  his  own  pace,  some  subjects  were  able  to  complete 
more  problems  than  others. 

The  two  subjects  who  performed  in  the  morning  session  responded  to  the 
questions  in  the  context  of  the  one  basic  initial  tactical  scenario. 

However,  for  the  four  subjects  who  performed  in  the  afternoon  session,  the 
updated  scenario  was  presented  after  most  of  the  questions  were  answered 
with  respect  to  the  initial  scenario.  Subjects  were  then  presented  with 
a  number  of  the  same  questions  a  second  time  and  asked  to  now  respond  in 
light  of  the  new  scenario.  Since  the  tactical  data  (i.e.,  the  overlays 
showing  unit  positions,  appended  information  parameters  such  as  unit 
capability,  etc.)  in  the  new  scenario  were  different  from  that  in  the 
initial  scenario,  subjects  had  to  request  a  new  sequence  and  configuration 
of  overlays  in  order  to  respond.  Of  course,  the  names  of  the  overlays  and 
symbol  portrayal  techniques  used  in  each  were  the  same  in  both  scenarios. 

In  this  manner  participants  proceeded  to  answer  the  questions  and  receive 
new  ones  until  the  conclusion  of  the  test.  Following  the  test,  participants 
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were  asked  to  complete  a  user  opinion  questionnaire.  This  questionnaire 
presented  several  statements  about  the  actual  and  potential  use  of  the 
selective  call-up  displays;  for  each  statement, the  participant  was  asked 
to  Indicate  his  level  of  agreement/disagreement  on  an  11-point  response 
scale.  Upon  completion  of  the  questionnaire,  a  debriefing  was  conducted 
with  all  participants  so  that  additional  conments  could  be  elicited. 

Experimental  Materials  (Symbol  Overlays) 

Study  participants  were  permitted  to  select  any  of  twenty  overlays, 
individually  or  In  combination,  to  look  at  while  formulating  an  answer  to 
each  of  the  tactical  questions  presented.  Table  2  describes  each  of  the 
overlays,  providing  a  description  and  example  of  their  content  as  well  as 
the  portrayal  method  and  color  used  by  the  overlay.  Several  overlays  and 
overlay  configurations  are  Illustrated  in  Fiqures  1  throuqh  12;  it 
should  be  noted  that  although  many  of  the  overlays  used  colors  other  than 
black  to  code  tactical  information,  all  Illustrations  provided  here  are 
monochromatic.  Each  individual  overlay  contains  very  specific  information 
on  only  one  dimension;  and  by  virtue  of  this  technique  of  overlay  develop¬ 
ment,  the  overlays  could  be  easily  combined  with,  as  well  as  deleted  from, 
each  other.  For  example,  overlay  #1  (Figure  1.)  shows  only  the  core 
symbols  for  friendly  maneuver  units  in  their  respective  positions.  Overlay 
#4  (Figure  2)  contains  only  the  unit  designators  for  the  friendly  maneuver 
units.  When  overlays  #1  and  #4  are  combined,  the  result  is  illustrated  in 
Figure  3  which  portrays  each  friendly  maneuver  unit  together  with  its 
associated  designators.  Overlay  #6  (Figure  4),  as  another  example, 
contains  capability  information  (i.e.,  the  name  of  the  current  major 
weapon)  associated  with  each  friendly  maneuver  unit;  when  overlay  #6  is 
combined  with  overlay  #1  (i.e.,  the  core  symbols),  the  result  would  be  as 
shown  in  Figure  5;  when  overlay  #6  is  overlayed  together  with  overlays 
#1  and  #4,  the  result  would  be  as  shown  in  Figure  6.  Figure  7  shows 
overlay  #1  together  with  overlay  #8,  i.e.,  the  friendly  maneuver  units  and 
their  associated  strength  portrayed  by  perimeter  density  coding. 


TABLE  2 

INDEX  TO  SYMBOLOGY  OVERLAYS 


^■v,r  Aoes  d°' 


FIGURE  1 
OVERLAY  1: 


frifnoly  maneuver  units 

(ACTUAL  OVERLAY  IS  IN  BLUE) 


FIGURE  2 
OVERLAY  4: 


UNIT  DESIGNATORS  OF  FRIENDLY  MANEUVER  UNITS 
(ACTUAL  OVERLAY  IS  IN  BLUE) 
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FIGURE  3 

OVERLAYS  1  AND  4  COMBINED:  FRIENOLY  MANEUVER  UNITS  AND  THEIR  DESIGNATORS 


FIGURE  4  . 

OVERLAY  6:  CAPABILITY  OF  FRIENDLY  MANEUVER  UNITS 
(ACTUAL  OVERLAY  IS  IN  BLACK) 
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FIGURE  5 

OVERLAYS  1  AND  6  COMBINED:  FRIENDLY  MANEUVER  UNITS  AND  THEIR  CAPABILITY 


,  DRAGON 

•Gs*'7 


DRAGON 


FIGURE  6 

OVERLAYS  1,  4,  AND  6  COMBINED:  FRIENDLY  MANEUVER  UNITS  AND  THEIR 
DESIGNATORS  AND  CAPABILITY 
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FIGURE  7 

OVERLAYS  1  AND  8  COMBINED:  FRIENDLY  MANEUVER  UNITS  AND  THEIR  STRENGTH 


FIGURE  8 

OVERLAYS  12  AND  20  COMBINED:  ENEMY  MANEUVER 
INDICATORS 


17 


122H0U 


122H0W 


120H0RT 


120MORT 


FIGURE  9 

OVERLAY  15:  CAPABILITY  OF  ENEMY  SUPPORT  UNITS 
(ACTUAL  OVERLAY  IS  IN  BLACK) 


122H0H 

120M0RT 
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FIGURE  10. 

OVERLAY  19:  WEAPON  RANGE  INDICATOR  OF  ENEMY  SUPPORT  UNITS 
(ACTUAL  OVERLAY  IS  IN  RED) 


FIGURE  11 

OVERLAYS  15  AND  19  COMBINED:  CAPABILITY  AND  RANGE  INDICATORS  FOR  ENEMY 
SUPPORT  UNITS 
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FIGURE  12 


OVERLAYS  1,  4,  6,  15,  AND  19  COMBINED:  FRIENDLY  MANEUVER  UNITS  AND  THEIR 
DESIGNATORS  AND  CAPABILITY  WITH  CAPABILITY  AND  RANGE  IWICATORS  FOR  ENEMY 
SUPPORT  UNITS 
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Figure  8  through  11  present  similar  examples  for  enemy  units.  Figure  8 
combines  overlays  #12  and  #20  to  portray  enemy  maneuver  units  and  their 
movement  indicators  (arrows);  Figure  9  displays  overlay  #15  by  itself, 
depicting  the  capability  (current  major  weapons)  of  the  enemy  support 
units;  Figure  10  gives  overlay  #19  by  itself,  showing  the  weapon  range 
indicators  for  the  enemy  support  units;  and  Figure  11  combines  the 
information  in  overlays  #15  and  #19.  As  a  final  illustration.  Figure  12 
portrays  the  combination  of  overlays  1,  4,  6,  15,  and  19.  The  level  of 
clutter  evident  from  the  combination  of  only  five  overlays  (Figure  12 
Is  worth  noting,  although  the  actual  computer  generated  picture  may  have 
been  somewhat  easier  to  Interpret  because  it  involved  the  use  of  three 
different  colors  (blue,  black,  and  red). 

Various  methods  of  portrayal  and  coding  were  used  in  the  overlays.  Table 
2  provides  a  key  to  the  methods  of  portrayal  and  coding  employed  for  each 
component  of  unit  Information;  the  rows  of  the  table  are  organized  according 
to  general  unit  type  (maneuver,  support,  or  service  support)  and  affiliation 
(friend!  or  enemy).  The  corresponding  overlay  numbers  appear  inside  the 
table,  and  these  numbers  match  those  listed  in  Table  2  .  The  method  of 
unit  identification  (e.g..  Figures  1  and  8)  was  taken  from  the 
military  symbology  standard  (FM  21-30,  Military  Symbols),  with  color 
coding  used  to  distinguish  enemy  and  friendly  units  (red  and  blue  is  the 
military  convention  for  this  distinction).  Friendly  designators  (e.g., 
Figure  2  )  were  also  portrayed  according  to  convention. 

The  remaining  dimensions  of  unit  information— namely,  capability,  (i.e., 
current  major  weapon)  strength,  range,  and  movement— were  each  graphically 
displayed  in  a  manner  that  is  not  represented  in  FM  21-30.  The  method 
used  in  each  case  to  append  the  respective  supplementary  information  to 
the  core  symbol  was  guided  by  a  number  of  concerns  including  common  (but 
nonstandardized)  practices  used  by  military  personnel,  meaningful  ness, 
avoidance  of  conflict  with  conventionally  displayed  information  (e.g., 
unit  designators),  legibility,  and  discriminabil ity.  The  latter  criteria 
are  of  particular  importance  because  of  the  need  for  simultaneous  overlays 
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TABLE  3 

CODING  KEY  TO  SYMBOLOGY  OVERLAYS 


UNIT 

Unit  ♦ 
Information 

CLASSIFICATION 

Method 
of  * 
Portrayal 

Friendly 

Enemy 

Support 

Friendly 

Enemy 

Service 

Friendly 

Support 

Enemy 

fl  cation 


Ba 


Designators 

Capability 

Strength 

A1 pha- 
Numerlcs 

Alpha- 

Numerics 

Perimeter 

Density 
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to  be  displayed  and  viewed.  For  example,  since  both  the  capability  and 
strength  of  units  would  likely  be  desired  to  appear  together,  the  former 
is  coded  one  way  (by  alphanumerics,  e.g..  Figures  4  and  9  )  and  the 
latter  is  coded  ahother  way  (by  perimeter  density,  with  the  number  of 
sides  filled  in  reflecting  different  levels  of  percent  strength,  e.g., 

Figure  7).  The  color  black  was  selected  to  portray  capability  and 
strength  because  of  discriminabllity  concerns;  in  particular  perimeter 
sensity  would  have  been  difficult  to  Identify  had  it  been  coded  the  same 
color  (i.e.,  blue)  as  the  unit  symbol.  The  portrayal  of  weapon  range 
indicators  (e.g..  Figure  10)  employs  vectors  drawn  to  scale  to  reflect 
the  actual  range  of  the  principal  weapon  system  of  each  unit  and  arrows 
are  used  to  indicate  moving  units  (e.g..  Figure  8),  while  not  a 
convention  of  FM  21-30,  vectors  and  arrows  are  commonly  used  by  military 
personnel  and  were  therefore  employed  in  this  investigation. 

In  the  study,  eighteen  different  symbols  were  employed,  with  each  symbol 
representing  a  different  type  of  military  unit.  As  shown  in  Table  4, 
the  symbols  are  classified  into  three  major  types— namely  maneuver  units 
(4  symbols),  support  units  (8  symbols),  and  service  support  units  (6 
symbols),  the  table  represents  a  key  to  each  symbol's  meaning  and  this 
key  was  provided  to  the  experimental  subjects.  These  core  unit  symbols 
appeared  on  five  overlays;  the  total  number  of  symbols  on  each  overlay,  and 
the  frequency  with  which  each  unique  symbol  appeared,  was  determined  by 
realistic  military  consideration  appropriate  to  the  given  tactical  scenario. 
For  the  initial  scenario,  the  symbol  distributions  were  as  follows:  overlay 
#1  (FR  MNVR)  contained  12  symbols— 7  infantry,  3  armor,  and  2  infantry 
command  post;  overlay  #2  (FR  SUPPORT)  contained  14  symbols— 7  artillery, 

2  mortar,  and  5  reconnaissance  observation  post;  overlay  #3  (FR  SVC  SPT) 
contained  6  symbols— 1  for  each  type  of  service  support  unit;  overlay  #12 
(EN  MNVR)  contained  12  symbols— 8  mechanized  infantry,  3  armor,  and  1 
mechanized  infantry  command  post;  overlay  #13  (EN  SUPPORT)  contained  11 
symbols— 5  artillery,  2  air  defense  artillery,  and  1  each  of  four 
remaining  types  of  enemy  support  units).  Overall,  there  were  a  total 
of  55  symbols— 32  friendly  symbols  (overlays  #1,  #2,  and  #3)  and  23  enemy 
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symbols  (overlays  #12  and  #13).  For  the  updated  second  scenario,  the  number 
of  units  and  their  distributions  on  the  overlays  were  roughly  the  same  as 
in  the  initial  scenario,  although  some  modifications  In  frequencies  were 
made  to  reflect  the  new  tactical  situation. 

Participants  were  permitted.  In  this  preliminary  investigation, to  select 
among  three  different  display  backgrounds;  and  each  overlay, or  overlay 
combination,  had  to  be  displayed  on  one  (and  only  one)  of  these  three 
backgrounds.  These  Included  a  standard  topographic  map,  a  blank  white 
background,  and  a  blank  black  background.  The  map  was  a  computer-generated 
digitized  gray  and  white  map  with  a  scale  of  approximately  1:88,000,  with 
grid  lines,  and  of  medium  detail  (Including  roadways,  waterways,  built-up 
areas,  vegetation,  but  no  contour  lines).  All  overlayed  symbol  information 
was  observable  on  the  map  background  and  on  the  blank  white  background. 
However,  the  appended  symbol  information  coded  in  black  (alphanumeric 
capability  information  and  perimeter- density  strength  information)  was 
masked  out  and  not  detectable  on  the  blank  black  background.  Thus,  when 
this  Information  was  requested  (i.e.,  overlays  6,  7,8,9,14,15,16,17),  either 
a  map  background  or  blank  white  background  was  necessary  in  order  for  the 
black-coded  information  to  be  seen. 

Data  Recording.  While  each  subject  performed  in  answering  each  question 
the  computer  kept  track  of  which  overlays  and  map  backgrounds  were  selected 
and  deleted  and  recorded  exactly  when  these  selections/deletions  were  made. 
This  data,  referred  to  as  the  "journal  record"  enabled  the  sequence  and 
timing  of  overlay  selection  to  be  determined  for  each  problem  solution. 

Table  4-5  provides  an  example  of  a  journal  record  for  one  subject's 
response  protocol  on  a  single  problem  (Question  B).  From  this  record, 
several  data  elements  could  be  derived  including  the  number  of  different 
overlays  selected  (14  in  this  case),  the  number  deleted  (2  in  this  case), 
which  overlays  and  map  backgrounds  were  selected/deleted,  and  when  each 
selection/deletion  was  made  (both  in  sequence  and  in  time).  Also,  the 
duration  that  each  overlay  appeared  on  the  screen  could  be  computed;  for 
example,  overlay  #7,  the  fourth  overlay  called  up,  was  in  view  for  40 
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seconds  from  the  time  It  was  selected  (1:26)  until  the  time  It  was  deleted 
(2:06).  Fran  this  journal  record,  It  Is  evident  that  the  user  built  up 
a  large  set  of  overlays,  moving  from  friendly  overlays  to  enemy  overlays, 
and  kept  most  In  view  through  the  completion  of  the  problem  solution. 

With  regard  to  map  background,  the  blank  white  background  was  maintained 
during  the  first  two-thirds  of  the  problem  time  (and  during  the  period 
In  which  overlays  were  manipulated);  then  the  user  switched  to  a  back¬ 
ground  map  for  the  remainder  of  the  time. 

Results  and  Discussion 


The  primary  purpose  of  this  results  and  discussion  section  is  to  use  the 
available  pilot  data  as  a  vehicle  for  demonstrating  some  representative 
types  of  descriptive  analyses  that  can  be  performed  on  overlay-selection 
data.  In  addition,  we  want  to  show  how  the  results  of  such  analyses  can 
be  meaningfully  interpreted.  We  wish  to  emphasize,  however,  that  because 
of  the  limited  size  of  the  data  sample  we  have  no  intention  of  drawing  any 
conclusions,  or  deriving  any  specific  guidelines,  from  these  data  concerning 
the  utilization  or  effectiveness  of  the  selective  overlay  system.  The 
preliminary  investigation  was  conducted  to  collect  information  so  that 
intelligent  refinements  could  be  made  to  the  operational  dynamics  (e.g., 
man-machine  Interaction,  etc.)  of  the  selective  call-up  system,  as  well 
as  to  the  experimental  design,  before  the  comprehensive  experiment  is 
undertaken.  This  experiment  Is  to  be  conducted  early  in  the  second  year 
of  contractual  work,  and  the  findings  in  that  study,  based  on  more  formal 
and  extensive  statistical  analysis,  will  be  used  to  formulate  more  conclu¬ 
sive  judgments  about  the  workings  and  value  of  the  innovative  graphic 
techniques  under  investigation.  The  content  of  this  section  covers  a 
discussion  of  the  use  of  the  overlays  including  a  schematic  analysis, 
performance  time  data,  background-map  use,  and  user  opinion  ratings. 

In  accordance  with  the  original  design  of  the  preliminary  study,  it  was 
planned  that  participants  would  be  able  to  answer  five  tactical  questions 
far  the  initial  scenario  (involving  a  friendly  defensive  posture)  and  the 
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same  five  questions  applied  to  the  updated  (second)  scenario  (involving 
a  friendly  offensive  posture).  However,  this  objective  could  not  be 
achieved  mainly  because  of  certain  software  problems  (detected  and 
remedied  In  the  morning  session)  and  because  of  performance  time  constraints. 
For  one  thing,  the  briefing  and  demonstration  session  took  longer  than 
expected.  For  another,  some  test  participants  took  more  time  than  antici¬ 
pated  to  complete  each  question,  and  one  participant  was  unable  to  remain 
through  the  conclusion  of  the  experimental  session.  In  any  event,  fully 
recorded  data  were  made  available  for  a  total  of  five  questions.  Four 
questions  were  responded  to  by  each  of  four  participants;  three  of  these 
questions  were  answered  In  the  context  of  the  first  scenario  (Al,  C,  and 
F)  and  one  question  was  answered  in  the  context  of  the  second  scenario 
(A2).  And,  the  remaining  question  (B,  first  scenario)  was  responded  to 
by  all  six  participants.  The  results  and  discussion  to  follow  Is  there¬ 
fore  based  on  the  performance  data  collected  for  these  questions. 

Use  of  Overlays.  One  important  issue  addressed  by  this  research  is  the 
frequency  or  relative  degree  of  use  for  each  of  the  candiate  symbology 
overlays  across  tactical  questions.  In  particular,  data  on  frequency  of 
overlay  use  may  guide  decisions  concerning  the  need  for  overlay  design 
refinements,  overlay  consolidation,  and  the  possible  deletion  of  certain 
overlays  from  the  developing  system.  Participants  varied  considerably  with 
regard  to  the  number  of  overlays  that  they  called  up;  this  number  ranged 
from  2  to  11  per  problem  with  a  mean  of  6.  Table  6  provides  the  frequency 
of  use  of  each  of  the  overlays  for  each  tactical  question  for  which  data 
are  available;  the  overlays  (coded  by  number  and  name)  are  listed  in 
descending  order  according  to  the  total  frequency  of  use.  The  cell  entries 
in  the  table  can  also  be  interpreted  in  terms  of  the  number  of  participants 
who  reviewed  an  overlay.  For  example,  overlay  #1  was  viewed  by  all  four 
participants  in  their  responses  to  questions  Al,  A2,  C,  and  F  and  was 
viewed  by  five  out  of  six  participants  in  response  to  question  B. 
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As  can  be  seen  from  inspection  of  Table  6,  the  participants  placed  greatest 
emphasis  on  friendly  and  enemy  maneuver  units  and  the  attributes  of  these 
units.  Such  concern  with  a  dynamic  view  of  the  battlefield  situation  was 
observed  previously  by  Geiselman  and  Samet  (1980)  and  Samet  and  Gelselman 
(1981)  In  conjunction  with  summaries  of  tactical  Intelligence  data.  The 
equal  frequency  spread  between  friendly  and  enemy  overlays  Is  of  Interest- 
In  both  the  set  of  10  most  frequently  called  and  10  least  frequently  called 
overlays,  half  are  friendly  and  half  are  enemy.  Also  some  overlays  (e.g., 

#1,  #12,  #20,  #4)  appear  to  be  Important  across  all  five  tactical  questions, 
whereas  others  (e.g.,  #11,  #13,  #7)  tend  to  be  called  up  only  for  certain 
questions.  Only  one  of  the  overlays  (#17,  enemy  strength  for  support  units) 
was  never  called  up.  Whatever  the  case,  because  of  the  pilot  nature  of 
this  Investigation  and  limited  number  of  participants/questions/scenarios, 
no  decision  can  be  made  from  this  data  with  regard  to  refinements  to  the 
existing  set  of  candidate  overlays. 

Schema  Analysis.  A  suitable  methodology  for  examining  the  participants' 
overlay  selection  protocols  is  provided  by  an  application  of  "schema 
theory",  from  cognitive  psychology.  This  methodology  has  been  applied 
successfully  In  another  context  by  Geiselman  and  Samet  (1980)  to  analyze 
intelligence  summaries  prepared  by  tactical  officers.  Schema  theory  holds 
that  the  comprehension  of  any  information  is  affected  by  past  knowledge, 
which  is  organized  as  a  mental  template  for  interpreting  and  understanding 
new  Information.  Since  a  set  of  schemata  would  exist  for  tactical  opera¬ 
tions  data  in  the  minds  of  skilled  officers,  our  participants  are  a 
resource  for  understanding  which  overlays  are  most  Important  and  in  which 
order  they  are  most  appropriately  viewed.  A  schema  of  the  participants' 
overlay  selection  behavior  can  be  represented  as  a  two-dimensional  plot 
of  the  overlays.  In  this  case,  the  vertical  dimension  is  selection 
percentage  (the  percentage  of  the  participants  who  selected  a  given 
overlay)  and  the  horizontal  dimension  is  average  study  position  percentile, 
where  a  study  position  percentile  Is  equal  to  [(sequential  position  of 
call  up  for  an  overlay/total  number  of  overlays  called  up)  X  100].  Thus, 
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the  vertical  dimension  reflects  how  Important,  on  average,  an  overlay  Is 
considered  to  be  In  answering  the  question,  and  the  left-to-rlght  dimensions 
reflects  the  preferred  order,  on  average,  of  selection  of  the  overlays. 

As  with  most  statistical  techniques,  the  reliability,  validity,  and 
general izabillty  of  the  results  of  a  schema  analysis  would  tend  to  Increase 
with  the  size  of  the  data  sample.  For  the  purpose  of  this  discussion, 
schema  analysis  is  applied  to  a  very  small  sample  of  data.  However,  our 
Intention,  at  this  time,  is  to  Illustrate  the  application  of  an  analytical 
method  and  not  to  draw  specific  conclusions  about  the  use  of  tactical 
symbology  overlays.  In  particular,  we  want  to  demonstrate  how  the  data 
to  be  obtained  in  the  comprehensive  follow-up  experiment  can  be  treated 
and  represented  as  well  as  to  indicate  what  types  of  inferences  might  be 
drawn  from  such  data.  Nevertheless,  it  might  be  mentioned  that  the  useful¬ 
ness  of  a  schema  analysis  derived  from  the  data  of  only  six  subjects  has 
been  previously  demonstrated  In  the  human  factors  literature  (Geiselman  & 
Samet,  1980;  Samet  &  Geiselman,  1981). 

To  Illustrate  the  application  of  schema  analysis  to  the  utilization  of  the 
overlays,  two  tactical  questions  were  selected  for  indepth  study  and 
exposition  here,  questions  A1  and  B.  Four  participants  answered  question 
A1  and  six  participants  answered  question  B.  These  questions  were  chosen 
because  they  were  questions  for  which  some  of  the  participants  gave  answers 
that  were  clearly  superior  to  those  given  by  the  remaining  participants, 
and  corresponding  differences  were  evident  in  the  overlay  selection  data. 
Thus,  differences  in  the  schema  could  be  studied  as  a  function  of  the 
quality  of  performance.  A  panel  of  three  military  experts  rated  the 
participants'  answers  to  the  tactical  questions  on  two  dimensions,  namely 

accuracy  and  completeness.  The  accuracy  dimension  reflected  the  correctness 
of  the  answer,  while  the  completeness  dimensions  pertained  to  the  rationale 
and  thoroughness  of  the  answer.  Ten  point  "goodness"  scales  were  used  for 
each  dimension,  and  questions  were  scored  on  the  basis  of  total  points 
assigned  by  the  experts. 
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Question  A1  [I.e.,  question  A  (Table  1)  applied  to  the  Initial  tactical 
scenario]  asked  for  a  recomnendatlon  as  to  where  the  regiment's  tank 
reserves  should  be  re-located,  given  the  friendly-offensive  scenario.  An 
example  of  an  answer  that  was  considered  to  be  "good"  by  the  panel  of 
experts  appears  below. 

"I  would  recormend  that  the  regimental  tank  reserve  he 
employed  in  the  area  between  Alpha  Company  (A-l/7)  and  Bravo 
Company  (B-l/7).  The  reasoning  for  this  answer  is  firstly , 
that  the  enemy  attack  appears  to  be  focused  in  the  area 
occupied  by  Bravo  Company.  Secondly t  Alpha  Company  is  at 
half  strength,  Bravo  Company  is  currently  at  3/4  strength 
and  undergoing  further  attrition ,  whereas  Charlie  Company 
( C-l/7 )  is  at  full  strength." 

Figure  13  illustrates  the  application  of  schema  analysis  to  the  overlay 
selection  results  for  question  A1  as  a  function  of  the  quality  of  the 
answers  given.  The  top  panel  refers  to  the  participants  whose  answers 
were  considered  "good"  by  evaluators  and  the  bottom  panel  refers  to  the 
participants  whose  answers  were  considered  less  adequate,  i.e.,  "poor". 

In  each  panel,  a  two-dimensional  outline,  or  schema  is  presented  of  the 
participants'  overlay  selection  behavior.  There  are  two  overlays  which 
both  of  the  "good"  performers  selected  for  viewing  that  the  "poor" 
performers  did  not,  namely  overlay  #14  (capability  of  enemy  maneuver 
units)  and  overlay  #8  (strength  of  friendly  maneuver  units).  These  two 
overlays  would  appear  to  be  important  for  deciding  where  the  reserves 
should  be  emplaced.  It  is  also  apparent  that  the  two  "poor"  performers 
disagreed  with  each  other  concerning  overlay  selection  more  so  than  the 
two  "good"  performers.  The  "good"  performers  agreed  on  8  of  10  overlays 
selected,  whereas  the  "poor"  performers  agreed  on  only  3  of  8  overlays 
selected. 

A  preferred  order  for  viewing  the  overlays  is  specified  by  the  left-to- 
right  relationships  in  the  figure.  The  "good"  performers  chose  to  examine 
friendly  and  enemy  maneuver  units  first  (overlays  #1  and  #12),  with  one  of 
the  two  participants  also  requesting  the  overlay  showing  the  friendly 
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SELECTION  PERCENTAGE  SELECTION 


FIGURE  13 

SCHEMATA  OF  OVERLAY  SELECTIONS  FOR  QUESTION  A1 
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support  units  (overlay  #2).  Attention  was  then  directed  toward  the 
capability  of  the  enemy  maneuver  units  (#14),  the  unit  designators  for 
the  friendly  maneuver  units  (#4),  the  strength  of  both  the  friendly  (#8) 
and  enemy  (#16)  maneuver  units,  and  the  movement  of  the  enemy  maneuver 
units  (#20).  Finally,  one  of  the  two  "good"  performers  called  up  the 
designators  for  the  friendly  support  units  (#5)  and  both  participants 
called  the  overlay  showing  enemy  range  indicators  (#19).  The  "good" 
performers  viewed  the  designators  for  the  friendly  maneuver  units  (#4) 
much  earlier  in  the  study  period  than  did  the  "poor"  performers,  while 
viewing  the  enemy  range  indicators  (#19)  much  later.  It  Is  of  Interest 
that  none  of  the  participants  requested  to  see  the  unit  designators  early- 
on  together  with  the  unit  symbols  themselves.  This  suggests  that  such 
information  may  be  optional  and  that  it  could  constitute  clutter  at 
certain  stages  of  the  battlefield  comprehension  process. 

Question  8  (see  Table  1)  required  the  participants  to  recommend  whether 
or  not,  as  well  as  where,  any  of  the  friendly  artillery  batteries  should 
be  relocated.  Data  were  available  from  all  six  pilot  test  participants 
for  this  question.  From  the  six  answers  obtained  for  Question  B,  four 
answers  were  judged  by  the  expert  raters  to  be  superior  to  the  other  two. 
Again,  differences  in  overlay  selection  were  evident,  as  seen  in  the 
performance  schemata  presented  in  Figure  14.  The  schema  for  the  "poor" 
performers  suggests  a  considerable  variation  in  overlay  selection.  In 
contrast,  there  was  considerable  agreement  concerning  overlay  selection 
in  the  "good"  performance  group.  Specifically,  all  four  members  of  this 
group  agreed  on  the  first  five  overlays  selected,  and  two  members  of  the 
group  selected  the  same  ten  out  of  twelve  overlays.  The  most  important 
overlays  in  terms  of  selection  order  for  question  B  were  as  follows: 
friendly  maneuver  units  (#1),  friendly  support  units  (#2)  and  their 
designators  (#5),  enemy  maneuver  units  (#12),  and  the  weapon  range  of 
friendly  support  units  (#11),  respectively.  While  both  "poor"  performers 
selected  three  of  these  overlays,  namely  #2,  #5,  and  #11,  only  one  of  these 
participants  looked  at  overlay  #1  and  the  other  participant  looked  at  #12. 
Tactically,  both  of  these  overlays  are  highly  relevant,  since  artillery 
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should  be  placed  in  such  a  way  as  to  support  the  friendly  maneuver  units 
engaged  in  combat.  Thus,  the  "poor"  performers  might  not  have  placed  their 
emphasis  on  the  most  relevant  aspects  of  the  battlefield  for  making  their 
decisions  concerning  the  artillery  placement. 

Additional  insight  on  the  manner  in  which  the  overlays  were  used  can  be 
gained  by  comparing  overlay-selection  behavior  across  different  tactical 
questions.  For  example,  the  overlays  selected  for  question  A1  differed 
from  those  selected  for  question  B.  A  total  of  ten  overlays  were  selected 
by  the  "good"  performers  for  answering  Question  Al,  and  twelve  were  selected 
for  answering  question  B.  Three  overlays  were  unique  to  question  Al  and 
five  were  unique  to  question  B.  Six  of  the  eight  unique  overlays  pertained 
to  unit  capability  and  strength.  For  question  Al  strength  and  capability 
overlays  were  concerned  only  with  maneuver  units,  while  for  question  B 
strength  and  capability  information  was  requested  for  support  units.  As 
would  be  expected,  these  selections  map  directly  to  the  content  of  the 
questions,  one  concerned  with  friendly  reserve  employment  and  the  other, 
with  artillery  use. 

Thus,  the  pilot  data,  although  based  on  a  limited  sample,  are  Interpretable, 
meaningful,  and  suggest  what  could  be  learned  from  a  more  complete  set  of 
overlay-selection  protocols  In  response  to  differ*  ‘  tactical  questions. 

In  particular,  the  patterns  of  overlay  selections  qood"  performers 
appear  to  be  internally  consistent  as  well  as  different  from  those  of  the 
"poor"  performers.  An  examination  of  such  distinctions  may  enhance  our 
understanding  of  how  experienced  personnel  comprehend  a  battlefield 
situation  most  effectively,  especially  with  regard  to  the  use  of  tactical 
displays. 

Time  Data.  The  length  of  time  that  each  overlay  Is  maintained  in  view 
during  a  problem-solving  sequence  is  a  dependent  measure  of  principal 
interest  in  this  research,  especially  since  time-on-screen  provides  an 
additional  index  of  an  overlay's  usefulness.  Unfortunately,  however, 
because  of  the  operational  characteristics  of  the  computer  software  used 


37 


T 


in  the  pilot  investigation,  the  time  data  collected  in  this  investigation 
are  not  very  useful.  In  particular,  the  study  time  variable  appears  to 
have  been  confounded  by  an  unwillingness  on  the  part  of  the  participants 
to  use  the  overlay  deletion  option.  That  is,  once  an  overlay  was  called 
up,  it  was  rarely  deleted.  This  could  have  resulted  from  a  chosen 
strategy  where  the  display  is  "built  up"  one  overlay  at  a  time;  but  more 
likely,  a  system  limitation  Influenced  the  participants'  behavior.  Namely, 
It  took  considerably  more  time  and  effort  to  delete  any  given  overlay  from 
the  display  than  to  call-up  a  new  overlay.  This  limitation  will  be 
corrected  prior  to  the  next  test  of  the  system.  In  any  case,  the  study-time 
data  obtained  from  the  pilot  work  are  not  meaningful  in  the  sense  that  they 
were  generally  determined  by  the  order  in  which  the  overlays  were  selected. 

Nevertheless,  an  illustration— if  only  a  crude  one— of  what  can  be  learned 
from  time  data  would  be  useful  to  the  present  discussion  of  the  results 
for  the  preliminary  investigation.  This  illustration,  based  on  performance 
on  only  one  tactical  question,  considers  average  view  time  across  all 
overlays  rather  than  an  average  view  time  for  each  specific  overlay  (the 
latter  measure,  given  enough  subjects  and  questions,  would  of  course  be 
preferable).  Table  7  provides  the  available  overlay  view-time  data  for 
four  test  participants  in  response  to  question  Al,  organized  according  to 
the  "good"  and  "poor"  performers.  For  each  participant,  the  first  column 
indicates  the  total  amount  of  time  spent  in  viewing  all  his  selected 
overlays  prior  to  answering  the  question.  The  second  column  indicates  the 
total  number  of  overlays  viewed  during  the  solution  process  for  this 
problem.  The  f  nal  column  gives  the  average  view  time  per  overlay;  however, 
this  number  Is  to  be  Interpreted  in  light  of  the  fact  that  the  number  of 
overlays  displayed  (at  any  one  time)  increased  monotonically  throughout 
the  problem.  Although  these  data  are  based  on  only  four  participants  and 
one  tactical  problem,  some  interesting  observations  are  possible.  For  one 
thing,  there  is  more  variability  between  the  two  "pnor"  performers  than 
between  the  two  "good"  performers  both  in  term-  ?f  time  to  answer  and  number 
of  overlays  used.  However,  the  mean  time  each  o  lay  was  viewed  (as 

computed  here)  was  roughly  the  same  for  each  participant.  This  result 
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suggests  that  the  sequence  of  overlays  called  up  and  their  respective 
content  may  have  a  greater  Impact  on  information  processing  performance 
than  the  display  durations  for  the  individual  overlays. 

Map  Use.  In  the  pilot  investigation,  test  participants  rarely  chose  to 
display  the  topographical  map  (in  contrast  to  a  blank  white  or  black 
screen)  as  a  backdrop  for  viewing  the  overlays.  In  fact,  only  one 
participant  chose  to  consistently  call  up  the  background  map  and  did  so 
in  the  process  of  answering  each  and  every  question.  However,  even  this 
participant  did  not  study  the  various  overlays  against  the  map  background. 
Rather,  he  called  up  the  map  toward  the  end  of  the  problem  solving  sequence 
after  most  of  the  problem  time  had  already  been  spent  in  viewing  the 
overlays  against  a  blank  background.  A  number  of  reasons  may  account 
for  why  the  map  background  was  so  Infrequently  called  includuing  the 
following:  (a)  participants  were  already  quite  familiar  with  the  one 
geographical  area  of  Interest  (namely.  Camp  Pendleton  where  they  were 
stationed)  which  represented  the  single  background  identified  by  the 
tactical  scenario;  (b)  exact  grid  coordinates  for  any  point  represented 
on  any  overlay  could  be  determined  directly  by  the  user  by  positioning  a 
cursor  and  querying  the  computer  (that  is,  specific  location  information 
could  be  obtained  without  the  use  of  the  map);  (c)  the  resolution  and 
quality  of  the  map  was  not  perceived  to  be  high  enough  to  make  it  useful 
for  answering  the  specific  tactical  questions  presented;  and  (d)  the  detail 
on  the  map,  although  limited,  contributed  sufficient  information  clutter 
in  conjunction  with  the  overlays  so  as  to  make  the  display  of  the  map 
undesirable.  Thus,  if  use  of  the  background  map  is  to  be  made  more  relevant 
in  future  system  evaluations,  certain  changes  in  both  map  content  and 
display  format  will  probably  be  required. 

User  Opinion.  As  described  In  the  method  section  above,  the  participants— 
after  performing  in  the  preliminary  investigation— provided  scaled  agree¬ 
ment/disagreement  ratings  in  response  to  a  set  of  statements  about  the 
actual  and  potential  use  of  the  selective  call-up  displays.  These 
statements  are  listed  in  the  left-hand  column  of  Table  8.  The  number 
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TABLE  8 

RESULTS  FOR  USER  QUESTIONNAIRE 
(N-6) 


Statement  About  the  .Use 
of  Call-Up  Displays 

(1)  _ is  cumbersome  and  adds  to  the 

already  considerable  information 
overload  experienced  in  the  opera¬ 
tional  environment. 

(2)  . . .encourages,  user  to  explicitly 
identify,  relevant  information  needs 
and  to  evaluate  them  systematically. 

(3)  ...helps  user  focus  on  those  elements 
of  the  tactical  situation  of  primary 
concern  for  addressing  a  given  task. 

(4)  . . .  encourages  user  to  think  critically 
and  realistically  about  problems  and 
prospects  for  implementation  of 
plausible  alternatives. 

(5)  ...encourages  user  to  consider  a 
wide  range  of  options  or  possible 
action  alternatives. 

(6)  ...makes  the  user  more  confident  in 
evaluating  options  for  tactical 
planning  and  anslysis. 

(7)  ...contributes  to  the  educational 
process  with  respect  to  good 
procedures  for  processing  tactical 
information. 

(8)  ...is  a  realistic  goal  for  the 
near  future. 

(9)  ...will  require  more  expertise  from 
the  typical  user  than  is  likely  to  be 
normally  available  in  the  operational 
environment. 

(10)  ...can  be  expected  to  increase  as  users 
become  more  familiar  with  its  capabili¬ 
ties  in  an  operational  environment. 


Mean  Rating 
(0  -  Strongly  Disagree 
10  -  Strongly  Agree) 


8.3 

7.7 

7.0 


6.7 

6.8 

6.3 

8.5 

5.0 


9.0 
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In  the  right-hand  column  Is  the  mean  rating  assigned  by  the  six  participants 
to  the  statement  on  a  scale  from  0  (strongly  disagree)  to  10  (strongly 
agree).  As  is  evident  from  the  ratings,  the  users  viewed  the  system  in 
a  very  favorable  light.  For  the  two  negative  statements  (statements  1  and 
9  in  Table  8),  the  level  of  agreement  was  low  (3.5  and  5.0).  For  the 
remaining  eight  positive  statements,  the  level  of  agreement  was,  in  general, 
reasonably  high  (averaging  7.5).  In  particular,  user  opinion  emphasized 
the  beneficial  effects  of  this  type  of  task-adaptive  graphic  capability 
for  Improving  information  processing  and  decision  making  performance  in 
a  tactical  environment;  in  addition,  the  potential  utility  of  such  an 
automated  system  was  evaluated  positively.  These  results  are  encouraging 
in  light  of  the  fact  that  the  participants  responded  on  the  basis  of  only 
a  limited  exposure  to  a  very  preliminary,  unrefined  system  with  certain 
operational  limitations  that  are  to  be  improved  upon  in  the  next  iteration 
of  system  evaluation. 

2.2.4  Conclusion.  In  summary,  the  preliminary  investigation  provided 
a  considerable  amount  of  useful  data.  In  particular,  these  data  lead  to 
a  number  of  important  suggestions  with  regard  to  software  modifications 
that  will  make  use  of  the  selective  call-up  and  deletion  options  for  the 
overlays  easier  and  more  effective.  These  changes  relate  to  several  factors 
including  the  reorganization  of  the  display  (i.e.,  menu)  of  overlay  options, 
the  operational  manner  in  which  a  displayed  overlay  can  be  deleted  from 
the  screen,  the  color  of  the  blank  background,  and  the  format  for  output 
of  the  "journal  record"  (i.e.,  the  performance  data).  In  addition,  certain 
refinements  will  be  made  with  respect  to  the  experimental  design  and 
procedures  including  the  use  of  an  appropriate  number  of  subjects,  more 
extensive  practice  trials,  an  experimental  session  long  enough  to  provide 
wi thin-subject  data  on  several  questions  for  each  of  the  two  tactical 
scenarios.  More  detail  concerning  the  variety  of  proposed  modifications 
is  provided  in  the  next  chanter  under  Experiment  I. 
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